Using rasterization to identify traffic signal devices

ABSTRACT

Systems and methods are provided for identifying and representing a traffic signal device. The method includes determining a location and orientation of the vehicle and receiving a real world image. The method further includes analyzing information about the vehicle&#39;s location and environment and using this information and the vehicle&#39;s orientation to generate a raster image illustrating an approximation of a view of the real world image, including one or more traffic signal devices. Additionally, the method includes providing the real world image and the raster image as inputs to a neural network to classify a traffic signal device in the real world image as the primary traffic signal device and determine a set of coordinates indicating a location of the primary traffic signal device, generating a classified real world image which includes a bounding box indicating the set of coordinates, and receiving the classified real world image.

RELATED APPLICATION AND CLAIM OF PRIORITY

This patent document claims priority to, and is a continuation of, U.S.patent application Ser. No. 16/817,708, filed Mar. 13, 2020, which willgrant Dec. 28, 2021 as U.S. Pat. No. 11,210,571.

BACKGROUND

The present disclosure relates to traffic signal device identificationand, in particular, to using rasterization to identify traffic signaldevices.

Traffic signal devices are critical to safe driving. They signal when itis the safe, legal, and appropriate time for vehicles to pass or entercertain intersections or other regions. For this reason, autonomousvehicles require the ability to accurately detect and classify trafficsignal devices. This information is used to accurately instruct theautonomous vehicle as to which maneuvers are allowed by the vehicle.

Various techniques are available for identifying traffic signal devices.One such technique is the use of deep learning methods. Deep learningtechniques and methods have the advantage of being largely data-drivenand able to integrate information from a wide range of example imagesinto a single classifier. While many architectures of deep learningmodels follow a common pattern of successive layers that reduce thedimensionality of an input image, the methods by which these modelsrepresent their inputs and outputs vary substantially.

Traffic signal devices may be detected using standard object detectiontechniques, and, in the field of self-driving or autonomous cars orother vehicles, deep neural networks are often used for object detectionand classification. In a typical object detection task, a neural networkis configured to locate an arbitrary number of objects in a scene. Sincetraffic signal devices are often mapped, at a particular location theremay be some prior knowledge. Examples include what kinds or types ofobjects should be present in the scene and what their rough sizes are.

Viewing and isolating a traffic signal device is not necessarily astraight-forward process. Multiple traffic signal device faces may bevisible from a particular location and angle, making it difficult toisolate which traffic signal device is the correct traffic signaldevice. Therefore, for at least these reasons, a better method ofefficiently and accurately identifying and isolating a face of a correcttraffic signal device is needed.

SUMMARY

According to an aspect of the present disclosure, a method foridentifying and representing a traffic signal device is provided. Themethod includes, determining a location and orientation of a vehicle bya geographic location system of the vehicle and receiving a real worldimage that includes one or more traffic signal devices by an imagesensor. The method further includes, by a processor, analyzinginformation about one or more features of an environment at the locationof the vehicle, using the analyzed information and the orientation ofthe vehicle to generate a raster image of the environment, providing thereal world image and the raster image as inputs to a neural network toclassify one of the traffic signal devices in the real world image as aprimary traffic signal device and determine a set of coordinatesindicating a location of the primary traffic signal device in the realworld image, generating a classified real world image which includes abounding box indicating the determined set of coordinates, andreceiving, from the neural network, the classified real world image. Theraster image may be an approximation of a view of the real world imagecaptured by the image sensor of the vehicle when positioned at thelocation and orientation of the vehicle, and the raster image mayinclude one or more traffic signal devices that appear in the view, oneof which being labeled as the primary traffic signal device.

According to various embodiments, the set of coordinates is a set offour coordinates in a two-dimensional plane.

According to various embodiments, the classified real world imagefurther includes one or more sets of coordinates indicating a locationof any secondary traffic signal devices in the real world image.

According to various embodiments, in the raster image, any secondarytraffic signal devices are each represented by a mask corresponding to alocation of the secondary traffic signal device.

According to various embodiments, the method further includes applying acolor channel to each mask in the raster image, in which the colorchannel applied for each mask distinguishes the primary traffic signaldevice from each of the secondary traffic signal devices.

According to various embodiments, the bounding box indicates a discreteregion of pixels in the classified real world image.

According to various embodiments, the geographic location systemincludes a Global Positioning System device.

According to another aspect of the present disclosure, a system foridentifying and representing a traffic signal device is provided. Thesystem includes a geographic location system of a vehicle configured todetermine a location and orientation of a vehicle, a transceiverconfigured to send and receive digital information, and an image sensorconfigured to capture a real world image that includes one or moretraffic signal devices. The system further includes a processorconfigured to analyze information about features of an environment atthe location of the vehicle and the orientation of the vehicle togenerate a raster image of the environment, provide, using thetransceiver, the real world image and the raster image as inputs to aneural network to classify one of the traffic signal devices in the realworld image as a primary traffic signal device and determine a set ofcoordinates indicating a location of the primary traffic signal devicein the real world image, generate a classified real world image whichincludes a bounding box indicating the determined set of coordinates,and receive, from the neural network, the classified real world image.The raster image may be an approximation of a view of the real worldimage captured by the image sensor of the vehicle when positioned at thelocation and orientation of the vehicle, and the raster image mayinclude one or more traffic signal devices that appear in the view, oneof which labeled as the primary traffic signal device.

According to various embodiments, the set of coordinates is a set offour coordinates in a two-dimensional plane.

According to various embodiments, the classified real world imagefurther includes one or more sets of coordinates indicating a locationof any secondary traffic signal devices in the real world image.

According to various embodiments, in the raster image, any secondarytraffic signal devices are each represented by a mask corresponding to alocation of the secondary traffic signal device.

According to various embodiments, the processor is further configured toapply a color channel to each mask in the raster image, in which thecolor channel applied for each mask distinguishes the primary trafficsignal device from each of the secondary traffic signal devices.

According to various embodiments, the bounding box indicates a discreteregion of pixels in the classified real world image.

According to various embodiments, the geographic location systemincludes a Global Positioning System device.

According to yet another aspect of the present disclosure, a system foridentifying and representing a traffic signal device is provided. Thesystem includes a vehicle including a geographic location systemconfigured to determine a location and orientation of the vehicle, animage sensor configured to receive a real world image that includes oneor more traffic signal devices, and a computer-readable storage mediumwhich includes one or more programming instructions. The one or moreprogramming instructions, when executed, cause the vehicle to analyzeinformation about one or more features of an environment at the locationof the vehicle, generate a raster image of the environment using theanalyzed information and the orientation of the vehicle, provide thereal world image and the raster image as inputs to a neural network toclassify one of the traffic signal devices in the real world image as aprimary traffic signal device and determine a set of coordinatesindicating a location of the primary traffic signal device in the realworld image, generate a classified real world image which includes abounding box indicating the determined set of coordinates, and receive,from the neural network, the classified real world image. The rasterimage may be an approximation of a view of the real world image capturedby the image sensor of the vehicle when positioned at the location andorientation of the vehicle, and the raster image may include one or moretraffic signal devices that appear in the view, one of which labeled asthe primary traffic signal device.

According to various embodiments, the set of coordinates is a set offour coordinates in a two-dimensional plane.

According to various embodiments, the classified real world imagefurther includes one or more sets of coordinates indicating a locationof any secondary traffic signal devices in the real world image.

According to various embodiments, in the raster image, any secondarytraffic signal devices are each represented by a mask corresponding to alocation of the secondary traffic signal device.

According to various embodiments, the computer-readable storage mediumfurther includes one or more programming instructions that, whenexecuted, cause the vehicle to apply a color channel to each mask in theraster image, in which the color channel applied for each maskdistinguishes the primary traffic signal device from each of thesecondary traffic signal devices.

According to various embodiments, the bounding box indicates a discreteregion of pixels in the classified real world image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example of a system for identifying and classifying trafficsignal devices, in accordance with various embodiments of the presentdisclosure.

FIG. 2 is an example of an image of various traffic signal devices andthe environment around the various traffic signal devices, in accordancewith the present disclosure.

FIG. 3 is an example of a rasterized representation of various trafficsignal devices representing the positions of each of the faces ofvarious traffic signal devices, in accordance with the presentdisclosure.

FIG. 4 shows a flowchart of an example of a method for identifying andclassifying a traffic signal devices, in accordance with variousembodiments of the present disclosure.

FIG. 5 illustrates a block diagram of example hardware that may be usedto contain or implement program instructions according to variousembodiments of the present disclosure.

DETAILED DESCRIPTION

As used in this document, the singular forms “a,” “an,” and “the”include plural references unless the context clearly dictates otherwise.Unless defined otherwise, all technical and scientific terms used hereinhave the same meanings as commonly understood by one of ordinary skillin the art. When used in this document, the term “comprising” (or“comprises”) means “including (or includes), but not limited to.” Whenused in this document, the term “exemplary” is intended to mean “by wayof example” and is not intended to indicate that a particular exemplaryitem is preferred or required.

In this document, when terms such “first” and “second” are used tomodify a noun, such use is simply intended to distinguish one item fromanother, and is not intended to require a sequential order unlessspecifically stated. The term “approximately,” when used in connectionwith a numeric value, is intended to include values that are close to,but not exactly, the number. For example, in some embodiments, the term“approximately” may include values that are within +/−10 percent of thevalue. Also, terms such as “top” and “bottom”, “above” and “below”, andother terms describing position are intended to have their relativemeanings rather than their absolute meanings with respect to ground. Forexample, one structure may be “above” a second structure if the twostructures are side by side and the first structure appears to cover thesecond structure from the point of view of a viewer (i.e., the viewercould be closer to the first structure).

An “electronic device” or a “computing device” refers to a device thatincludes a processor and memory. Each device may have its own processorand/or memory, or the processor and/or memory may be shared with otherdevices as in a virtual machine or container arrangement. The memorywill contain or receive programming instructions that, when executed bythe processor, cause the electronic device to perform one or moreoperations according to the programming instructions. Examples ofelectronic devices include personal computers, servers, kiosks,mainframes, virtual machines, containers, gaming systems, televisions,and mobile electronic devices such as smartphones, personal digitalassistants, cameras, tablet computers, laptop computers, media playersand the like. In a client-server arrangement, the client device and theserver are electronic devices, in which the server contains instructionsand/or data that the client device accesses via one or morecommunications links in one or more communications networks. The servermay be a single device or a collection of devices that are distributedbut via which processing devices and/or memory are shared. In a virtualmachine arrangement, a server may be an electronic device, and eachvirtual machine or container may also be considered to be an electronicdevice. In the discussion below, a client device, server device, virtualmachine or container may be referred to simply as a “device” forbrevity.

In this document, the terms “memory,” “memory device,” “data store,”“data storage facility” and the like each refer to a non-transitorydevice on which computer-readable data, programming instructions or bothare stored. Except where specifically stated otherwise, the terms“memory,” “memory device,” “data store,” “data storage facility” and thelike are intended to include single device embodiments, embodiments inwhich multiple memory devices together or collectively store a set ofdata or instructions, as well as individual sectors within such devices.

In this document, the terms “processor” and “processing device” refer toa hardware component of an electronic device that is configured toexecute programming instructions. Except where specifically statedotherwise, the singular term “processor” or “processing device” isintended to include both single-processing device embodiments andembodiments in which multiple processing devices together orcollectively perform a process.

In this document, “electronic communication” refers to the transmissionof data via one or more signals between two or more electronic devices,whether through a wired or wireless network, and whether directly orindirectly via one or more intermediary devices. Devices are“communicatively connected” if the devices are able to send and/orreceive data via electronic communication.

Referring now to FIG. 1 , a system 100 for identifying and classifyingtraffic signal devices 130 is provided.

Traffic signal devices 130, 145 convey driving information to driversvia the one or more signal elements 135 located on the face 140 of thetraffic signal devices 130, 145. The traffic signal elements 135 aredynamic in that they can be changed between at least two states totransmit traffic instructions to one or more drivers, and differenttypes of signal elements 135 may be present in a single traffic signaldevice. Examples of traffic signal elements 135 may include, forexample, a red light, a yellow light and a green light. Other examplesinclude lights with directional arrows (such as arrows pointing left orright), other symbols (such as a symbol of a person walking), or words.In each of these examples, each light can be switched between and offstate and an on state.

It is common for the environment around a traffic signal device 130 tobe fairly complicated. For example, multiple types of lighted objects,such as other street lights, crosswalk indicators, and even othertraffic signal devices 145, may be in view from a vehicle 105, as shownin the image of multiple traffic signal devices in FIG. 2 , which ispresented by way of example, showing an instance with multiple trafficsignal devices. In order to identify which of these elements is thecorrect, or primary, traffic signal device, the present disclosuredescribes a system 100 and means of isolating the correct, or primary,traffic signal device 130.

Traffic signal devices 130, 145 are mapped, and information provided inthe mappings of the traffic signal devices includes thethree-dimensional location of the face 140 of each of the traffic signaldevices 130, 145, the three-dimensional location of the signal elements135 within each of the faces 140 of the traffic signal devices 130, 145,and the type and color of each signal element 135. This information,combined with the location, orientation, and camera calibrations of thevehicle 105, means that it is possible to know the location andorientation in each camera image of every traffic signal device 130 ofinterest with moderate certainty.

In the example shown in FIGS. 1 and 3 , there are three traffic signaldevice faces that project into the image in close proximity to oneanother. In this situation, it is potentially confusing as to whichtraffic signal device should be classified as the correct, or primary,traffic signal device. However, it is of critical importance for thetraffic signal device classifying system 100 to correctly classify theprimary traffic signal device 130 face 140, since classification of anincorrect face could result in the vehicle 105 taking the incorrect, andlikely unsafe, action.

According to various embodiments, the system 100 includes a vehicle 105.According to various embodiments, the vehicle 105 is traveling on a road110. It is noted, however, that any suitable path may be implemented.

The vehicle 105 may include a computer vision system 115 configured toreceive a digital image of a traffic signal device 130. The computervision system 115 may include a camera for imaging one or more trafficsignal devices 130. The vehicle may include a geographic location system160 configured to determine a location and orientation of the vehicle105. The geographic location system 160 may include a Global PositioningSystem device. It is noted, however, that other forms of geographiclocation may additionally, or alternatively, be used.

The traffic signal device shown in FIG. 1 includes various signalelements 135, represented as circular lights. However, the features ofeach of the elements 135 may be any of various element features such as,for example, a green light, a yellow light, a red light, a circularlight, a left arrow light, a right arrow light, a forward arrow light, aflashing yellow light, a flashing red light, and/or any other suitabletraffic signal element features. It is further noted that the trafficsignal device 130 may include any suitable number of signal elements135, having various positions on the face of the traffic signal device130. The traffic signal elements 135 correspond to a designated lightfixture configured to transmit traffic instructions to one or moredrivers. The “real world image” is a digitally captured photograph of aview from the perspective of the vehicle 105.

The vehicle 105 may further include a transceiver 120 configured to sendand receive digital information from a remote server 155 via a wiredand/or wireless connection such as, for example, through the cloud 150,wherein the vehicle 105 and the remote server 155 are in electroniccommunication with each other. The vehicle 105 may further include aprocessor 125. The processor 125 may be configured to receive, using thetransceiver 120, information pertaining to features of an environment atthe location of the vehicle 105, and use the information and theorientation of the vehicle 105 to generate a raster image of theenvironment. It is noted that the processor 125 may be a standaloneprocessor 125, the vehicle's 105 processor 125, and/or the remoteserver's 155 processor 125. Data processed by the processor 125 may bedata received from the vehicle 105, received from the remote server 155,and/or a combination of data from the vehicle 105 and the remote server155. According to various embodiments, the vehicle 105 may include oneor more digital storage devices and some or all of the digitalinformation may be stored locally at the vehicle 105.

The processor 125 may be configured to represent the traffic signaldevice 130 as a raster image (such as that shown in FIG. 3 ) in whicheach traffic signal device 130, 145 is represented by a maskcorresponding to a location of each traffic signal device 130 in theraster image. According to various embodiments, the raster image is anapproximation of a view that will be captured by an image sensor 160 ofthe vehicle 105 when positioned at the location and orientation of thevehicle 105 and includes one or more traffic signal devices 130, 145that should appear in the view.

From a traditional object detection perspective in an image, errorspertaining to the confusion between two or more faces is difficult toaddress because there is relatively little prior known information aboutthe image. However, in regards to traffic signal devices 130, 145, thereis a great deal of information known about the location, position, andangle of traffic signal devices 130, 145. Using this information, andgiven the location and angle of a vehicle 105, it is possible to projectevery registered and mapped traffic signal device 130, 145 surroundingthe vehicle 105 into the coordinate system of the computer vision systemto create a rasterized representation of the scene that would be visibleto the computer vision system 115 of the vehicle 105.

The raster image forms a template to indicate the layout of a scenewhich includes various traffic signal devices 130, 145. The raster imageindicates the location of each face 140 of the traffic signal devices130, 145, as a rectangle (although any suitable shape may be used). Eachof the faces 140 of the traffic signal devices 130, 145 is representedby a discrete region of pixels. Each discrete region of pixels may berectangular in shape and/or any other suitable shape. The raster imageincludes information indicating which of the traffic signal devices 130,145 in the raster image is the primary traffic signal 130. According tovarious embodiments, this information is determined using location andorientation information pertaining to the vehicle 105, in addition toany mapping information pertaining to any traffic signal devices mappedin the area in the calculated view of the vehicle 105. It is noted,however, that other data may, in addition or alternatively, be used todetermine the primary traffic signal device 130 in the raster image.

As shown in FIG. 3 , the raster image forms a template to indicate thelayout of a scene which includes various traffic signal devices 130,145. The raster image shown in FIG. 3 indicates the location of eachface 140 of the traffic signal devices 130, 145, as a rectangle(although any shape may be used), and it indicates which face 140 (theface 140 of the primary traffic signal device 130) the network shouldpay attention to by rendering it with a unique mask. Any secondarytraffic signal devices 145 may also be represented with masks. Accordingto various embodiments, the primary traffic signal device 130 and anysecondary traffic signal devices 145 are assigned channels applied totheir respective masks. According to the embodiment shown in FIG. 3 ,the primary traffic signal device 130 is assigned to a blue colorchannel, and the secondary traffic signal devices 145 are assigned to ared color channel, although any suitable channel designation may beused.

According to various embodiments, the real world image and the rasterimage are provided as inputs to a neural network. According to variousembodiments, the real world image and the raster image are compared andthe raster image is used as a template to determine which traffic signaldevice is the primary traffic signal device 130 in the real world image.According to various embodiments, the comparison between the real worldimage and the raster image is used to determine a set of coordinates inthe real world image indicating the position of the primary trafficsignal 130 in the real world image. These coordinates may be used togenerate a classified real world image in which the primary trafficsignal device 130 is represented by a bounding box corresponding to theset of coordinates indicating the position of the primary traffic signaldevice 130 in the real world image. The set of coordinates may be of anysuitable type. For example, the set of coordinates may be a set of fourcoordinates in a two-dimensional plane.

According to various embodiments, the raster image (as shown in FIG. 3 )and a real world image of the scene are provided to the network asinputs and compared in order to determine which portion of the realworld image pertains to the primary traffic signal device 130.

According to various embodiments, when the raster image and the realworld image are provided to the network as inputs, the images arecompared, using the raster image as a template. Matching the real worldimage to the template, the primary traffic signal device 130 on the realworld image is determined, indicating to the network which face shouldbe used to report information pertaining to a state of each element 135in the primary traffic signal device 130. In addition to identifying theposition of each traffic signal device 130, 145 in the image, the rasterimage may be configured to additionally, or alternatively, provideinformation pertaining to the relative distance of one or more of thetraffic signal devices 130, 145 from the vehicle 105, the states andtypes of one or more elements of the primary 130 or secondary 145traffic signal devices, and/or any other suitable information.

Referring now to FIG. 4 , a method 400 for identifying and classifying aprimary traffic signal device is illustratively depicted.

While driving, a vehicle may come across one or more traffic signaldevices. Depending on the location and orientation of the vehicle, oneor more traffic signal devices may be in view of the vehicle. At 405,using a geographic location system, a geographic location and anorientation of the vehicle are determined and, at 410, using aprocessor, information pertaining to features of an environment at thelocation of the vehicle are received. The information pertaining to theenvironment may include any mapping information pertaining to anytraffic signal devices in view from the vehicle. It is noted that theprocessor may perform one or more of the steps described herein inmethod 400.

Using a computer vision system, which includes an image sensor (e.g., acamera, etc.), a real world image of a view from the vehicle is capturedand received, at 415. The real world image may include a plurality oftraffic signal devices in view from the vehicle.

Using the received information and the orientation of the vehicle, theprocessor, at 420, generates a raster image of the environment from theviewpoint of a computer vision system of the vehicle. The raster imageis an approximation of the view captured by the image sensor of thecomputer vision system when positioned at the location and orientationof the vehicle, and the raster image may include one or more trafficsignal devices that should appear in the view.

At 425, a color channel may applied to each mask in the raster image, inwhich the color channel applied for each mask distinguishes the primarytraffic signal device from each of the secondary traffic signal devices.According to some embodiments, location and classification informationpertaining to the traffic signal devices in the raster images, such aswhich traffic signal device in the raster image is the primary trafficsignal device may be provided as input to a neural network. According tosome embodiments, a position of a primary traffic signal device and anysecondary traffic signal devices may be classified on the raster image,wherein a mask is applied to the primary traffic signal device and anysecondary traffic signal devices in the raster image. This informationmay include environmental and orientation information.

At 430, the real world image and the raster image are provided as inputto the neural network, wherein the raster image functions as a templateand, at 435, is compared against the real world image in order todetermine, at 440, a location of the primary traffic signal device inthe real world image. This location may include a set of coordinatesindicating the location of the primary traffic signal device in the realworld image. These coordinates are then used, at 445, to generate aclassified real world image, which may include a bounding boxcorresponding to the set of coordinates indicating the location of theprimary traffic signal device in the real world image. The classifiedreal world image may further include one or more sets of coordinates forany secondary traffic signal devices in the real world image. At 450,the classified real world image is received from the neural network.

FIG. 5 depicts an example of internal hardware that may be included inany of the electronic components of an electronic device as described inthis disclosure such as, for example, an on-premises electronic device,an associate electronic device, a remote electronic device and/or anyother integrated system and/or hardware that may be used to contain orimplement program instructions. The vehicle described in this disclosuremay be an electronic device, including some or all of the componentsdescribed herein.

A bus 500 serves as the main information highway interconnecting theother illustrated components of the hardware. CPU 505 is the centralprocessing unit of the system, performing calculations and logicoperations required to execute a program. CPU 505, alone or inconjunction with one or more of the other elements disclosed in FIG. 5 ,is an example of a processor as such term is used within thisdisclosure. Read only memory (ROM) and random access memory (RAM)constitute examples of non-transitory computer-readable storage media1220, memory devices or data stores as such terms are used within thisdisclosure.

Program instructions, software or interactive modules for providing theinterface and performing any querying or analysis associated with one ormore data sets may be stored in the computer-readable storage media 510.Optionally, the program instructions may be stored on a tangible,non-transitory computer-readable medium such as a compact disk, adigital disk, flash memory, a memory card, a USB drive, an optical discstorage medium and/or other recording medium.

An optional display interface 515 may permit information from the bus500 to be displayed on the display 520 in audio, visual, graphic oralphanumeric format. Communication with external devices may occur usingvarious communication ports 525. A communication port 525 may beattached to a communications network, such as the Internet or anintranet. In various embodiments, communication with external devicesmay occur via one or more short range communication protocols.

The hardware may also include an interface 530, such as graphical userinterface, which allows for receipt of data from input devices such as akeyboard or other input device 535 such as a mouse, a joystick, a touchscreen, a remote control, a pointing device, a video input device and/oran audio input device.

The features and functions described above, as well as alternatives, maybe combined into many other different systems or applications. Variousalternatives, modifications, variations or improvements may be made bythose skilled in the art, each of which is also intended to beencompassed by the disclosed embodiments.

The invention claimed is:
 1. A method of representing a traffic signaldevice in an image, the method comprising: by a processor: receiving mapinformation that includes one or more traffic signal devices in anenvironment at a location of a vehicle; using the map information and anorientation of the vehicle to generate a raster image of theenvironment, wherein the raster image includes the one or more trafficsignal devices, one of which labeled as a primary traffic signal device;and providing a real world image of the environment at the location andthe raster image as inputs to a neural network to generate a classifiedreal world image which includes a bounding box indicating a location ofthe primary traffic signal device.
 2. The method of claim 1, wherein theneural network also generates a set of coordinates indicating thelocation of the primary traffic signal device.
 3. The method of claim 2,wherein the set of coordinates comprises four coordinates in atwo-dimensional plane.
 4. The method of claim 1, wherein the classifiedreal world image further indicates a location of a secondary trafficsignal device in the real world image.
 5. The method of claim 1,wherein, in the raster image, any secondary traffic signal devices areeach represented by a mask corresponding to a location of the secondarytraffic signal device.
 6. The method of claim 5, further comprisingapplying a color channel to each mask in the raster image, in which thecolor channel applied for each mask distinguishes the primary trafficsignal device from each of the secondary traffic signal devices.
 7. Themethod of claim 1, wherein the bounding box indicates a discrete regionof pixels in the classified real world image.
 8. A system forrepresenting a traffic signal device in an image, the comprising: aprocessor; a memory storing a neural network; and a computer-readablemedium storing programming instructions that are configured to instructthe processor to: receive map information that includes one or moretraffic signal devices in an environment at a location of a vehicle; usethe map information and an orientation of the vehicle to generate araster image of the environment, wherein the raster image includes theone or more traffic signal devices, one of which labeled as a primarytraffic signal device; and provide a real world image of the environmentat the location and the raster image as inputs to the neural network togenerate a classified real world image which includes a bounding boxindicating a location of the primary traffic signal device.
 9. Thesystem of claim 8, wherein the neural network is configured to generatea set of coordinates indicating the location of the primary trafficsignal device.
 10. The system of claim 9, wherein the set of coordinatescomprises four coordinates in a two-dimensional plane.
 11. The system ofclaim 8, wherein the classified real world image further indicates alocation of a secondary traffic signal device in the real world image.12. The system of claim 8, wherein, in the raster image, any secondarytraffic signal devices are each represented by a mask corresponding to alocation of the secondary traffic signal device.
 13. The system of claim12, further comprising additional program instructions that areconfigured to cause the processor to apply a color channel to each maskin the raster image, in which the color channel applied for each maskdistinguishes the primary traffic signal device from each of thesecondary traffic signal devices.
 14. The system of claim 8, wherein thebounding box indicates a discrete region of pixels in the classifiedreal world image.
 15. The system of claim 8, wherein: the processor is acomponent of the vehicle; and the vehicle also comprises a geographiclocation system configured to receive location data that the processoruses to select the map information.
 16. A non-transitorycomputer-readable medium storing programming instructions that areconfigured to instruct a processor to representing a traffic signaldevice in an image by: receiving map information that includes one ormore traffic signal devices in an environment at a location of avehicle; using the map information and an orientation of the vehicle togenerate a raster image of the environment, wherein the raster imageincludes the one or more traffic signal devices, one of which labeled asa primary traffic signal device; and providing a real world image of theenvironment at the location and the raster image as inputs to a neuralnetwork to generate a classified real world image which includes abounding box indicating a location of the primary traffic signal device.17. The computer-readable medium of claim 16, wherein the classifiedreal world image further indicates a location of a secondary trafficsignal device in the real world image.
 18. The computer-readable mediumof claim 16, wherein the programming instructions are also configured tocause, in the raster image, any secondary traffic signal devices each berepresented by a mask corresponding to a location of the secondarytraffic signal device.
 19. The computer-readable medium of claim 18,further comprising additional programming instructions that areconfigured to cause the processor to apply a color channel to each maskin the raster image, in which the color channel applied for each maskdistinguishes the primary traffic signal device from each of thesecondary traffic signal devices.
 20. The computer-readable medium ofclaim 16, wherein the bounding box indicates a discrete region of pixelsin the classified real world image.